Inter-Photon-Limited Videography


Andrew Xie1,2, Dongyu Du1,2, Sotiris Nousias3, David B. Lindell1,2 Kiriakos N. Kutulakos1,2,
1University of Toronto 2Vector Institute 3Purdue University

In this webpage we provide videos results, additional experiments, visual comparisons, and analyses that complement the main text of Inter-Photon-Limited Videography. We primarily compare with state-of-the-art self-supervised quanta video reconstruction methods bit2bit and Quanta Burst Photography (QBP).

Recovering Videos Across a Range of Inter-Photon Frequencies

Below, the elevator scene played back at 225fps (top) and 98.4kfps (bottom) as well as the corresponding binary frames captured by the SPAD512 single-photon camera. We show two playback speeds to illustrate the range of inter-photon frequencies captured: from the fast flickering elevator lights to the large motion of the jumping person to the slow moving elevator doors. Each frame has an exposure of 10us. We then simulated more difficult conditions with binomial thinning up to a factor of 1024.

Video Results

We supplement our main paper's results figure with additional video results.



We compare against QBP in their own dataset at two different inter-photon frequencies. At high inter-photon frequencies photons are not enough to accumulate and QBP reconstruction resembles the binary input.

We compare against bit2bit in their drill dataset. Bit2bit returns blurred images at higher inter-photon frequencies while our reconstruction is unaffected.

UWB fails to reconstruct a fan spinning even though it is well suited for periodic phenomena like this scene, since it neglects spatial correlations and the SNR at each pixel is poor.

Our method produces high-quality reconstructions across a wide range of inter-photon frequencies. We sucessfully recover the bullet's trajectory even with 300x fewer photons. We also show the correlation between inter-photon frequency with reconstruction difficulty and its effects.

We can also recover dynamic scenes with non-periodic motion.

We recover ultra-wideband videos using 10x fewer photons than previous work.

Baseline Comparisons Across Inter-Photon Frequency Ranges

We show binary frames captured by the SPAD512 array as well as video reconstructions of the optical flux. The scene is made more challenging via binomial thinning increasing fp. Our method retains more detail and better temporal consistency as the towards the inter-photon-limited regime.

Baseline Comparisons (Simulated)

We compare our method to QBP and bit2bit on various simulated scenes. Ground truth flux is obtained by temporally interpolating high-speed video sequences and reducing the DC level (and thus the inter-photon spectra) to match the low-light regime where single-photon imaging is most effective.